Bridging the Motivation Gap for Individual Annotators: What Can We Learn From Photo Annotation Systems?

The importance of incentives and socially based motivation for metadata generation should not distract attention entirely from the need to design tools for metadata generation that use every means available to maximize the efficiency and intrinsic motivation of the individual annotator. The popular application domain of (individual) photo management has recently given rise to a number of strategies and methods that can serve as a source of inspiration for the design of metadata generation support for the semantic web. This position paper offers a brief synthesis of relevant work that is intended to serve as a basis for the representation of this perspective at the Insemtive 2008 workshop. 1 Why Photo Annotation Is a Relevant and Instructive Scenario The problem of motivating contributions to a community-supported resource (of which the semantic web can be seen as an especially ambitious example) is often framed in terms of a contrast between the interests of an individual contributor and the interests of the group as a whole (see, e.g., [1]): If only people were as motivated to contribute to the semantic web as they are to their own personal knowledge bases, it would seem, the creation of metadata for the semantic web would thrive. While this perspective is valid and important, we would like to call attention to the fact that there can also be a major “motivation gap” when individuals are making similar contributions for their own benefit. Consequently, we also need to examine ways of closing the motivation gap that arise even when individuals are working for their own benefit. These methods can in turn also benefit the community-supported semantic web indirectly. More concretely, consider the familiar problem of adding metadata to photos: Since photos form a natural part of the semantic web as well as of many Web 2.0 systems, improving people’s motivation to add metadata to photos would constitute a contribution to the goals of this workshop. But even when an individual is managing their own personal photo collection, there is a challenging motivation gap: Having good metadata would make it much easier for the user to accomplish common tasks such as searching ⋆ The research described in this paper is being conducted in the context of the KnowDive project (http://disi.unitn.it/∼knowdive). The participation of the second author was supported by Vulcan, Inc. for photos that fit a particular description; but as has often been noted (see, e.g., [2]), few users get very far in adding such metadata, largely because of the time-consuming and tedious nature of the work that is involved. Because of the rapidly growing popularity and practical importance of digital personal photo collections, a good deal of research has been devoted in recent years to the problem of motivating and/or supporting untrained end users in adding metadata to their photos. Despite—or indeed because of—the differences between this scenario and the more general scenario of adding metadata for the semantic web, it is worthwhile to look closely at the successes that have been achieved in this area and to consider how they might be generalized. 2 Overview of Determinants of Successful and Motivating Photo Annotation Types of metadata that users often want to add to photos include (a) persons, objects, locations, and events depicted in the photos; and (b) information about the context in which the photo was taken (e.g., “just before sundown” or “just after the end of the championship football game”). It is often assumed that the photos already have accurate time and location stamps that can serve as input to automatic processing (though in fact such automatically generated metadata may be missing or incorrect for various reasons and may therefore need to be supplied by the user—a problem to which some of the metadata creation approaches discussed below can be applied). Figure 1 summarizes a number of the ideas that have emerged from recent work on interfaces that help users to add such metadata. Before discussing these points individually and illustrating them with reference to recent research, we will comment on them briefly. In terms of motivation, the overall approach taken in photo annotation systems for individuals is not based on external incentives or social mechanisms but rather on the provision of an intrinsically motivating experience for the individual user. Somewhat more concretely, the strategy is to optimize the relationship between (a) the cost to the user in terms of work done (in particular, tedious work) and; (b) the benefits in terms of enjoyable experiences, successful task performance, and visible improvements to the collection of items. In some ways, the most straightforward approach is to exploit external resources (see the bottom left-hand corner of the figure) that can straightforwardly generate new metadata on the basis of existing metadata (e.g., supplying the name of a town on the basis of GPS coordinates). But external resources may also serve as input to sophisticated algorithms that analyze the content of items, either suggesting metadata or at least grouping together items that appear (to the system) to belong in the same category. Since such algorithms do not in general perform perfectly, there is generally a user interface that is designed to enable the user to supply the necessary manual input with minimal effort and maximal enjoyment. The user input itself can be seen as a valuable resource, which includes both explicit annotation actions and naturally occurring actions that provide useful information although the user does not perform them specifically for the purpose of adding metadata.